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Unclassified 


PREFACE 


This  report  documents  the  planning  and  management  of  the  Air  Force  Skills  and  Maintenance 
and  Reacquisition  Training  Research  Program  (Project  SMART). 

This  research  was  conducted  under  Project  1123,  USAF  Flying  Training  Development,  James 
F.  Smith,  project  scientist;  Task  112302,  Flying  Training  Innovations,  Dr.  Bernell  J.  Edwards,  task 
scientist;  Work  Unit  11230236,  Skills  Maintenance  and  Reacquisition  Training  (Project  SMART). 
Dr.  Edward  E.  Eddowes,  principal  investigator. 

The  goals  of  this  project  are  to  develop  and  validate  comprehensive,  quantitative,  and  objective 
procedures  which  will  permit  the  Air  Force  to  manage  an  individualized  flying  training  program 
that  will  provide  acceptable  mission  readiness  at  minimum  cost.  This  effort  validates  previously 
developed  measures  of  skills  involved  in  the  pop-up  weapons  delivery  and  low  altitude  tactical 
formation  tasks  and  provides  evidence  of  the  validity  of  the  skills  measurement  approach  toward 
optimizing  pilot  mission  readiness. 


1 


TABLE  OP  CONTENTS 


Pv 


I.  Introduction .  5 

II.  Technical  Approach .  5 

III.  Method .  6 

IV.  Results .  6 

V.  Conclusion .  8 

References .  8 

Appendix  A:  Pop-Up  and  LATF  Self-Assessment  Forms .  9 

Appendix  B:  Identification  and  Definition  of  Critical  Skills  Involved 

in  Performing  the  Pop-Up  and  LATF  Tasks .  13 

Appendix  C:  Summary  Report:  Air-to-Surface  Weapon  Delivery 

Self-Assessment  Evaluation  Study .  18 

Appendix  D:  Summary  Report:  Low  Altitude  Tactical  Formation 

Self-Assessment  Evaluation  Study .  22 

Appendix  E:  Summary  Report:  TAC-Wide  Test .  24 


3 


SKILLS  MAINTENANCE  AND  REACQUISITION  TRAINING  RESEARCH  PROGRAM: 
TACTICAL  AIR  COMMAND  PRELIMINARY  EVALUATION 


1.  INTRODUCTION 


The  Skills  Maintenance  and  Reacquisition  Training  research  program  (Project  SMART)  is  based  on  a 
priority  Air  Force  (USAF/XOOT)  research  requirement  to  identify  and  define  critical  combat  skills  of 
mission-ready  aircrews  and  to  develop  a  user-oriented  technology  for  measuring  these  skills.  Project 
SMART  research  will  generate  the  quantitative  data  needed  to  determine  how  frequently  critical  flying 
skills  must  be  practiced  to  optimize  pilot  mission  readiness.  The  objective  of  Project  SMART  is  to  develop 
and  validate  comprehensive,  quantitative,  objective  procedures  for  the  efficient  management  of 
individualized  flying  training  which  will  provide  aircrew  mission  readiness  at  minimum  cost.  The 
objectives  of  the  Project  SMART  Tactical  Air  Command  (TAC)  Preliminary  Evaluation  were  (a)  to 
provide  an  early  evaluation  of  the  key  concepts  and  methods  of  all  phases  of  the  program,  (b)  to  identify 
and  define  selected  critical  flying  skills,  (c)  to  develop  procedures  for  measuring  these  skills,  and  (d)  to 
evaluate  and  refine  the  skill  measures. 


II.  TECHNICAL  APPROACH 


Previous  efforts  to  improve  flying  training  have  focuses  on  task  requirements.  Project  SMART, 
however,  focuses  on  identifying,  defining,  and  developing  procedures  for  measuring  the  critical  skills 
underlying  aircrew  mission  readiness.  Research  reported  by  Meyer,  Laveson,  Weissman,  and  Eddowes 
(1974)  and  by  Meyer,  Laveson.  Pape,  and  Edwards  (1978)  indicates  that  a  small,  manageable  number  of 
skills  can  be  identified  and  defined  which  cut  across  the  range  of  task  performances  that  are  required  of 
mission-ready  TAC  aircrews. 

Given  operationally  oriented  definitions  of  aircrew  mission-ready  skills,  along  with  feasible, 
meaningful  and  acceptable  aircrew  skill  measurement  techniques,  proficiency  levels  on  each  critical  skill 
can  be  determined  and  relevant  practice  can  be  prescribed  as  required  to  bring  such  skill  deficiencies  as 
are  detected  up  to  mission-ready  criterion  levels.  Consequently,  the  contents  of  each  sortie  flown  can  be 
justified  by  and  specified  on  the  basis  of  diagnostic  information  provided  by  the  skill  measurment 
procedures. 

Previous  research  systematically  addressing  Air  Force  continuation  training  has  been  minimal. 
Therefore.  Project  SMART  was  initiated  with  a  preliminary  evaluation  study  designed  to  develop  skill 
definition  and  measurement  procedures  based  on  study  of  a  small  sample  of  critical  flying  tasks.  During 
the  first  7  months  of  the  TAC  preliminary  evaluation,  research  efforts  were  aimed  at  developing, 
evaluating,  and  refining  skill  measurement  procedures  designed  to  assess  pilot  skills  exercised  in 
accomplishing  the  pop-up  weapons  delivery  (pop-up)  and  the  low  altitude  tactical  formation  (LATF) 
tasks.  These  measurement  procedures  were  developed  in  coordination  with  a  mission-ready  A-7  squadron 
and  an  F-4  combat  crew  training  squadron  through  a  series  of  iterative  refinements  based  on  pilot  self- 
assessment  records  collected  during  the  flying  training  operations  of  these  two  squadrons. 

Following  presentation  of  the  mid-term  progress  report  on  the  TAC  Project  SMART  Preliminary 
Evaluation  at  HQ  TAC  on  6  October  1978,  it  was  suggested  that  these  skill  measurement  procedures  be 
validated.  Consequently,  a  TAC-wide  test  of  skill  measurement  procedures  for  the  pop-up  and  LATF  tasks 
was  designed  and  implemented  during  January  and  February  1979. 


lit.  ME  HOD 


Participants.  Pilots  who  contributed  to  the  reiults  of  the  Preliminary  Evaluation  included  those  of 
the  354th  Tactical  Fighter  Squadron  (TFS),  Davis-Monthan  AFB;  the  311th  Tactical  Fighter  Training 
Squadron  (TFTS),  Luke  AFB;  the  4th  Tactical  Fighter  Wing  (TFW),  Seymour  Johnson  AFB;  the  23rd 
TFW,  England  AFB;  the  347th  TFW,  Moody  AFB;  the  354th  TFW,  Myrtle  Beach  AFB;  and  the  474th 
TFW,  Nellis  AFB. 

Materials.  Research  materials  consisted  of  the  LATF  and  the  pop-up  self-assessment  forms.  Copies  of 
the  pop-up  and  LATF  forms  are  shown  in  Appendix  A. 

Procedures.  The  TAC  preliminary  evaluation  was  initiated  with  the  cooperation  and  participation  of 
the  345th  TFS,  Davis-Monthan  AFB.  Semistructured  interviews  with  eight  fighter  pilots  of  the  354th  TFS 
were  recorded  and  analyzed.  Initial  interviews  were  conducted  to  identify  the  critical  tasks  involved  in 
fighter  mission  scenarios.  Having  identified  the  critical  tasks  to  be  studied,  the  research  pilots  analyzed 
each  task  into  its  real-time  components  and  described  the  behavioral  actions  required  to  perform  each 
component.  These  interviews  used  generalized  maneuver  patterns  as  guidelines.  Pilots  were  interviewed 
both  individually  and  in  groups. 

Critical  task  breakdown  summaries  were  constructed  using  the  data  obtained  from  these  interviews. 
The  summaries  were  reviewed  and  refined  in  further  interviews  to  obtain  a  consensus  on  their 
completeness  and  accuracy.  In  addition  to  the  original  research  pilots,  interview,  to  review  the  task 
breakdown  summaries  were  conducted  with  members  of  the  4444th  Operations  Squadron  (Operational 
Training  Development),  Davis-Monthan  AFB,  and  with  the  staff  of  the  Fighter  Weapons  School,  162nd 
Tactical  Fighter  Training  Croup  (TFTG),  Arizona  Air  National  Guard. 

Subsequent  interviews  focused  on  validation  of  the  skill  identification/definition  analyses. 
Interviews  were  supplemented  by  photographing  A- 7  pop-up  weapons  delivery  maneuvers  at  the  Gila 
Bend  Gunnery  Range  to  provide  additional  data  on  available  means  for  skill  measurement.  Many  of  these 
pop-up  passes  were  supplemented  by  audio  recordings  of  pilot  commentary  during  the  maneuver.  These 
data  served  as  the  basis  for  development  of  practical  techniques  for  assessing  the  critical  flying  skills 
identified  in  the  analyses  of  the  pop-up  and  LATF  tasks.  Once  developed,  the  skill  measurement 
procedures  were  used  in  routine  flying  operations  to  collect  data  to  use  in  evaluating  and  revising  them. 


IV.  RESULTS 

Identification  and  Definition  of  Critical  Tactical  Flying  Skills.  Following  completion  of  the  analyses 
of  the  pop-up  and  LATF  tasks,  the  tactical  taxonomy  of  Meyer  et  al.  (1978)  was  used  to  identify  the  flying 
skills  exercised  at  each  point  during  the  pop-up  maneuver  and  in  accomplishing  the  major  performance 
requirements  in  the  case  of  the  LATF  task.  The  skill  identified  and  their  definitions  were  reviewed  and 
refined  through  coordination  with  participating  pilots  of  the  354th  TFS  and  the  162nd  TFTG. 

Initially  six  skills  were  found  to  accommodate  the  performance  requirements  of  the  pop-up  and 
LATF  tasks;  Planning.  Recheck,  Discriminating,  Anticipating,  Deciding,  and  Controlling.  Their  brief 
definitions  are  shown  below: 

Planning  —  Describing  and  updating  mission  requirements. 

Recheck  —  Repetitive  seeking  and  filtering  of  information  from  within  and  outside  the 

cockpit. 
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Discriminating  —  Evaluating  fine  differences  among  cues/cue  patterns. 

Anticipating  —  Predicting  what  aircraft  control  actions  will  be  required. 

Deciding  —  Selecting  from  among  alternative  aircraft  control  actions. 

Controlling  —  Achieving  and  maintaining  a  series  of  aircraft  system  and  subsystem  states. 

The  analyses  of  the  pop-up  and  LATF  tasks  which  were  developed  to  support  the  identification  and 
definition  of  these  critical  flying  skills  are  presented  in  Appendix  B. 

Pop-Up  Measurement  Development.  Analysis  of  the  pop-up  disclosed  that  the  maneuver  consisted  of 
a  sequence  of  pilot  actions  required  to  control  the  aircraft  to  a  point  in  space  from  which  a  bomb  could  be 
released  so  as  to  hit  the  intended  target.  The  initial  analysis  was  summarized  by  stage  of  the  maneuver, 
and  the  major  skills  exercised  during  each  stage  were  identified.  A  self-assessment  form  was  developed  on 
which  pilots  rated  their  performance  on  each  stage  of  the  pop-up  thus  permitting  specification  of  the  skills 
involved.  Preliminary  evaluation  research  continued  with  collection  of  pop-up  weapons  delivery  skill  data 
for  use  in  refining  and  revising  the  self-assessment  procedures  and  forms.  Bombing  skill  data  were 
required  using  revised  versions  of  the  pilot  self-assessment  form  for  pop-up  maneuvers  flown  on  both 
tactical  and  controlled  ranges.  The  pop-up  self-assessment  data  were  processed  and  the  resulting  skill 
scores  were  analyzed  to  determine  their  relationship  with  bomb  impact  scores.  In  addition,  means  for 
processing  the  bombing  skill  data  for  easy,  rapid  interpretation  by  training  managers  were  explored. 

Pop-up  data  were  collected  systematically  in  a  study  using  F-4  (CCT)  B-course  students  of  the  311th 
TFTS.  In  this  case,  the  evaluation  forms  were  completed  by  instructor  pilots  (IPs).  Analyses  of  the  data 
confirmed  the  validity  of  the  measurement  techniques.  Another  similar  evaluation  was  performed 
subsequently  at  the  request  of  the  311th  TFTS.  Summary  reports  of  these  studies  are  presented  in 
Appendix  C. 

LATF  Measurement  Development.  The  analysis  of  low  altitude  tactical  formation  indicated  that  this 
task  involved  the  continuous  performance  of  four  major  components,  formation  position  keeping,  low 
altitude  flying,  mutual  support  (visual  lookout),  and  navigation.  Further  analysis  of  the  component 
performances  led  to  identification  of  the  major  skills  involved.  A  self-assessment  form  for  recording  pilot 
ratings  of  performance  on  the  LATF  components  provided  for  measurement  of  skills  in  the  same  manner 
employed  with  the  pop-up.  Skill  measurement  data  generated  through  the  use  of  the  pilot  self-assessment 
forms  were  studied  in  a  series  of  multiple  regression  analyses  to  provide  the  basis  for  revision  and 
refinement  of  the  skill  measurement  materials  and  procedures. 

LATF  data  were  collected  from  pilots  of  the  354th  TFS  and  subsequently  evaluated  to  determine  the 
utility  of  the  skill  measurement  procedures  for  the  LATF  task.  A  report  was  prepared  documenting  the 
successful  application  of  the  LATF  measurement  procedure  and  the  kinds  of  information  it  generated.  A 
summary  of  this  report  is  presented  in  Appendix  D. 

TAC-Wide  Tests.  The  pop-up  and  LATF  skill  measurment  materials  and  procedures  were  tested 
further  during  a  TAC-wide  test.  Five  tactical  fighter  wings  participated  in  the  test,  the  4th,  23rd,  354th, 
347th,  and  474th. 

During  the  TAC-wide  test,  a  total  of  1227  pop-up  and  LATF  forms  were  collected  from  participating 
pilots.  An  analysis  of  these  data  demonstrated  the  validity  of  the  measurement  procedures  and  their 
generalizability  in  applications  with  A-10  and  F-4  aircrew  personnel.  A  summary  report  of  the  TAC-wide 
teat  focusing  on  an  analysis  of  pop-up  data  is  presented  in  Appendix  E. 
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V.  CONCLUSION 


These  findings  indicate  that  Project  SMART  has  developed  a  methodology  for  identifying,  defining, 
and  measuring  critical  tactical  fighter  tasks  across  aircraft  systems.  In  addition,  the  methodology  is  also 
successful  in  identifying  critical  subtasks:  a  finding  that  can  have  substantial  diagnostic  value  for  training 
managers  in  specifying  priority  training  requirements. 

The  present  results,  however,  are  not  the  end  product  of  Project  SMART.  They  provide  a  substantial 
baseline  of  proven  research  procedures  and  a  data  base  to  support  and  stimulate  further  evaluation  and 
refinement  of  Project  SMART  objectives  and  techniques.  During  the  next  phase  of  the  research  program, 
critical  air  combat  maneuvering  and  air-to-ground  attack  skills  not  studied  in  Phase  1  will  be  identified 
and  defined  and  procedures  for  their  measurement  developed  and  validated. 
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APPENDIX  A:  POP-UP  AND  LATF  SELF-ASSESSMENT  FORMS 


POP-UP  EVALUATION  FORM 


PILOT  #: _  Squadron  #: _  RANGE  #: 

EVENT: _  BLOCK. _  DATE. 


PASS  # 

BOMB  SCORES 

1st  2nd  3rd 

4th 

Task  Evaluation 

COMMENTS/INDICATE  Pass  # 

1. 

Approach  to  PUP* 

2. 

PUP 

3. 

Climb  Leg 

4. 

Target  Acquisition 

5. 

Pull  Down  Point 

6. 

Apex 

7. 

Track  Point 

8. 

Bomb  Run 

9. 

Recovery 

10. 

Rtn  to  Ixjw  Level 

Legend:  E  —  Excellent  S  —  Satisfactory  M  —  Marginal  U  —  Unsatisfactory 
*Pl:P  —  pop-up  point 


i 

i 

V  j 

j 
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INSTRUCTIONS 


1.  COMPLETE  THE  PILOT  IDENTIFICATION  PORTION  OF  THE  FORM 

2.  RECORD  BOMB  SCORES. 

3.  GRADE  THE  TASK  EVALUATION  SECTION  AS  FOLLOWS: 

E  —  Excellent.  Task  performance  met  criteria  with  no  error  reflecting  an  unusually  high 

degree  of  ability;  no  compensations  were  required. 

S  —  Satisfactory.  Task  performance  met  criteria  with  minimal  error;  minimal 

compensation  were  required. 

M  —  Marginal.  Task  performance  met  criteria  with  error;  compensations  were  required  to 

salvage  the  pass  delivery. 

U  —  Unsatisfactory.  Task  performance  did  not  meet  criteria;  gross  errors  in  performance 

led  to  either  an  unsafe  or  aborted  pass. 

Task  evaluations  are  to  be  based  on  1)  proficiency  to  maneuver  the  aircraft,  2)  situation  awareness, 
3)  agressiveness  and  4)  survivability.  Any  item  graded  as  either  M  or  U  requires  an  appropriate 
explanation  under  the  Comments  section. 

The  following  indicates  those  requirements  identified  with  each  item  included  in  the  Task 
Evaluation  section: 

1.  Approach  to  PUP:  (a)  Acquisition  of  PUP;  (b)  Altitude  control;  (c)  Airspeed  control;  and  (d) 
Heading  control. 

2.  PUP:  (a)  Heading  correction;  (b)  "G"  applications;  (c)  Airspeed  correction;  and  (d)  timing/ 
distance  error. 

3.  Climb  Leg:  (a)  Climb  angle  corrections;  and  (b)  airspeed  corrections. 

4.  Target  Acquisition:  Self-explanatory 

5.  Pull  Down  Point:  (a)  Roll;  (b)  Airspeed  corrections;  (c)  “G”  application;  and  (d)  Altitude/position 
control. 

6.  Apex:  (a)  Pattern/position  correction;  and  (b)  Airspeed  corrections. 

7.  Track  Point:  (a)  Aim  off  point;  (b)  Roll  out;  (c)  Initial  wind  correction;  (d)  Angle,  azimuth,  and 
position  checks;  and  (e)  Initial  pipper  placement. 

8.  Bomb  Run:  (a)  Aiming  error  corrections;  (b)  Airspeed  control;  (c)  Exposure  time  control;  and  (d) 
Altitude,  azimuth,  and  dive  angle  corrections. 

9.  Recovery:  (a)  “G"  application;  (b)  jinking;  and  (c)  Altitude  and  timing  control. 

10.  Return  to  Low  Level:  (a)  Exposure  time;  and  (b)  Transition  to  low  level. 
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LATF  Individual  Performance  Evaluation 


Pilot  number _ Date _ Call  Sign. 


1. 

Visual  Lookout 

N/A 

U 

M 

S 

E 

X 

1  2  3 

4  5 

6  7 

2. 

Maintaining  Position: 

a.  Your  performance 

N/A 

U 

M 

S 

E 

X 

1  2  3 

4  5 

6  7 

b.  Element  member's  performance: 

N/A 

u 

M 

S 

E 

X  01234567 


3.  Rate  your  all  around  performance  during  each  mission  segment  below: 


a.  Takeoff  and  join-up: 

U 

M 

S 

E 

0 

1 

2 

3 

4 

5 

6 

7 

b.  Target  Ingress: 

U 

M 

S 

E 

0 

1 

2 

3 

4 

5 

6 

7 

c.  Time  on  target: 

N/A 

U 

M 

S 

E 

X 

0 

1 

2 

3 

4 

5 

6 

7 

d.  Rejoin  and/or  egress: 

U 

M 

S 

E 

0 

1 

2 

3 

4 

5 

6 

7 

e.  Recovery: 

N/A 

U 

M 

S 

E 

X 

0 

1 

2 

3 

4 

5 

6 

7 

4.  Give  Comfor  level:  CL _ 

_ Ft  AGL. 

5.  Comfort  Level  Rating: 

u 

M 

S 

E 

0 

1 

2 

3 

4 

5 

6 

7 

6.  Overall  Mission  Performance  Rating: 

U 

M 

S  E 

0  1 

2  3 

4  5  6  7 

7.  Familiarity  with  element  member: 

Unfamiliar 

Very  Familiar 

0  1 

2  3 

4  5  6  7 

Legend:  U— Unsatisfactory  M— Marginal  S— Satisfactory  E— Excellent 
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LATF  Individual  Performance  Evaluations 


DIRECTIONS 


For  the  mission  just  completed,  rate  each  of  the  following  skill  areas  from  0  to  7.  Assign  a  rating  of  0  to 
Unsatisfactory  performance.  Assign  a  rating  of  7  to  Excellent  performance. 

Definitions  of  Ratings 

Excellent:  A  rating  of  excellent  means  that  the  task  or  segment  was  performed  without  error  and  that  no 
corrective  actions  were  required. 

Satisfactory:  A  rating  of  satisfactory  means  that  the  task  or  segment  was  performed  with  minimal  error 
and  that  only  minor  corrective  actions  were  taken. 

Marginal:  A  rating  of  marginal  means  that  the  task  or  segment  was  performed  with  error  and  that 
corrective  actions  were  required. 

Unsatisfactory:  A  rating  of  unsatisfactory  means  that  the  task  or  segment  was  performed  with  large  or 
frequent  errors  and  that  major  corrective  actions  were  required. 


Skill  Areas 

1.  Visual  Lookout:  How  adequate  was  the  amount  of  time  you  spent  clearing  the  area  for  aerial  threats? 

2.  Maintaining  position:  How  well  did  you  (your  element  member)  maintain  position?  Did  you(he) 
signal  your(his)  intentions  clearly  and  respond  to  signals  appropriately? 

3.  Mission  Segments:  Rate  your  all-around  performance  on  each  segment  below.  Consider  all  factors 
relevant  to  each  segment. 

4/5.  Comfort  Level:  CL  is  the  lowest  altitude  at  which  you  can  detect  the  enemy  threat  and  perform  a 
defensive  reaction  requiring  intense  manuvering.  Give  your  opinion  of  your  CL  on  this  mission  in  FT 
AGL  and  rate  the  adequacy  of  your  CL  to  mission  requirements. 

6.  Overall  Rating:  Rate  the  quality  of  your  performance  on  all  phases  of  the  mission  as  a  whole. 

7.  Familiarity:  Compared  to  what  you  consider  optimum,  rate  the  amount  of  experience  you  have  had 
flying  with  your  element  member. 

Comments:  (Optional) 
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APPENDIX  B:  IDENTIFICATION  AND  DEFINITION  OF  CRITICAL  SKILLS 
INVOLVED  IN  PERFORMING  THE  POP-UP  AND  LATF  TASKS 


Pop-Up  Weapon  Delivery  Profile 


Pop-Up  Weapon  Delivery 
Skill  Identification 


Critical  Skills 

Sequence  of  Pilot  Actions  Planning  Recheck  Discriminating  Anticipating  Deciding  Controlling 


—  Before  Initial  Point  — 


Identify  initial  point  X  X 

('.heck  heading  X 

Set  armament  switches  X  X 

Review  attark  parameters  X  X 

Review  heading  change  X  X 

Prepare  to  hack  the  clock  X  X 

Review  defensive  formation 

over  target  X  X 

Review  change  of 
formation  at 

initial  point  X  X 

Check  inertial  guidance 

system  X  X 
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Pop-Up  Weapon  Delivery 
Skill  Identification  ( Continued ) 


Critical  SkilU 

Sequence  of  Pilot  Action*  Planning  Recheck  Discriminating  Anticipating  Deciding  Controlling 


—  At  Initial  Point  — 


Turn  to  preplanned 
heading 

Make  preplanned  airapeed 
change 

Make  preplanned  tactical 
formation  change 
Hack  clock 

Check,  heading,  airspeed, 
tactic*,  navigation 
Review  defensive  tactics 
Activate  master  arms 
switch 

Mentally  review  pop-up 
point  (PUP),  PUP 
parameters 

Check  terrain  appearance 
vs.  plan  expectations 
Review  mental  picture 
of  Urge!  area 
Review  formation  position 
Check  airspeed,  clock 
inertial 

Anticipate  pull-up 
Check  time 
Acquire  PUP 


X 

X 


X 


X 


X 

X 


X 


X 

X  X 

X 
X 


X 


X 


X 


X 


X 


X 

X 

X 

X 


X 


X 


X 

X 

X 

X 

X 


Co  to  full  power  at  PUP 
Pull  back  on  stick 
Monitor  climb  angle 
Set  a  climb  angle 
Check  for  threats 
Look  for  target 
(area  correlate) 

Check  altitude 
Anticipate  pull  down 
Visualise  track  point 
Recognise  deviations  from 
planned  parameters 
Analyse  available  turning 
room 

Continue  attack,  reposition 
to  acquire  target  or 
abort  attack 
Roll  in/pull  down 
Fly  aircraft  to  track  point 


—  At  Pop-Up  Point  — 

X 

X 

X 

X  X 

X 

X  X 

X  X 


X 


X 


X 


X 

X 

X 


X 

X 

X 

X 


X 


X 


14 


XXX 


f  . 


Pop-Up  Weapon  Delivery 
Skill  Identification  ( Continued) 


Critical  Skill* 

Sequence  of  Pilot  Action*  Planning  Recheck  Di (criminating  Anticipating  Deciding  Controlling 


Roll  out  at  track  point 
Fly  the  wire 
Acquire  target  now 
or  abort 

Recogniae  preplanned 
elease  parameters 
Maneuver  aircraft  to  put 
pipper  on  target 
Designate  when  pipper 
ia  on  target 
Pickle  (release  bomb) 


Turn,  look,  dive  to 
egress  target 
Check  for  threats 
Check  energy  level 
Establish  flight  integrity 
Maintain  mutual  support 
on  egress 

Egress  to  rendeavous 
point  if  aeparated 
from  formation 


—  At  Track  Point 


—  At  Releaae  Point 


—Exit  Target  Area 


Taak  Pha 


Before  initial  point 
At  initial  point 
At  PUP 

Turn  toward  target 
At  track  point 
At  releaae  point 
Exit  target  area 


Planning  Recheek  Discriminating  Anticipating  Deciding  Controlling 


LOW  ALTITUDE  TACTICAL  FORMATION 
DEFINITION  OF  TASK  REQUIREMENT 


FLYING  AT  LOW  ALTITUDE  A  PILOT  MUST  BE  ABLE  TO  FLY  IN 

FORMATION  COMFORTABLY  AND 
SAFELY  AT  ALTITUDES  BETWEEN 
300  AND  100  FEET  AGL. 

COORDINATION  OF  FORMATION  TASKS  MEMBERS  OF  A  TACTICAL 

FORMATION  MUST  MAINTAIN 
POSITION  TO  PERMIT  MUTUAL 
SUPPORT  DURING  AN  ATTACK  AND 
IN  THE  EVENT  OF  A  THREAT  TO  THE 
FLIGHT. 

SEARCHING  AN  AREA  BEHIND  AND 
ABEAM  OF  ONE'S  ELEMENT 
MEMBER  MUST  BE  PERFORMED 
WITHOUT  COMPROMISING  LOW 
ALTITUDE  FLYING  OR  POSITION  IN 
FORMATION. 

MEMBERS  OF  A  FLIGHT  MUST 
FOLLOW  AND  UPDATE  THE 
NAVIGATION  PLAN  TO  DELIVER  A 
SUCCESSFUL  ATTACK  AT  THE 
DESIGNATED  TIME  AND  TO  RETURN 
TO  BASE  SAFELY. 


Low  Altitude  Tactical  Formation 
Skill  Identification 


Critical  Skills 

Formation  Task  Requirement* 

Planning 

Recheck 

Diacriminating 

Anticipating 

Deciding 

Controlling 

Flying  al  low  altitude 

Coordination  of  formation 

X 

X 

X 

X 

X 

tanka 

X 

X 

X 

X 

X 

X 

Mutual  aupport 

X 

X 

X 

Navigation  to/from  target 

X 

X 

X 

X 

MUTUAL  SUPPORT 


NAVIGATION  TO/FROM  TARGET 


MEASURES  OF  LOW  ALTITUDE  TACTICAL  FORMATION  SKILL 
COMFORT  LEVEL  (CL): 

LOWEST  LEVEL  AT  WHICH  PILOT/FIGHTINC  UNIT  CAN  DETECT  AN  ENEMY  THREAT  AND 
PERFORM  A  DEFENSIVE  REACTION  REQUIRING  INTENSE  MANEUVERING 

MINIMUM  ALTITUDE  CAPABILITY  (MAC): 

LOWEST  ALTITUDE  AT  WHICH  A  PILOT/FIGHTING  UNIT  CAN  FLY  FOR  BRIEF  PERIODS  TO 
NEGATE  A  THREAT  FOLLOWING  DETECTION  BY  ENEMY. 

TACTICAL  COORDINATION 

PLANNING.  COORDINATING  AND  EXECUTING  APPROPRIATE  FORMATION  TACTICS. 
MUTUAL  SUPPORT 

PROPORTION  OF  SIMULATED  AIR  ATTACKS  ON  FORMATION  DETECTED. 

NAVIGATION 

ACCURACY  OF  NAVIGATION  TO  AND  FROM  TARGET. 


APPENDIX  C:  SUMMARY  REPORT:  AIR-TO-SURFACE  WEAPON 
DELIVERY  SELF-ASSESSMENT  EVALUATION  STUDY 


Pierce,  B.J.,  DeMaio,  J.,  Eddoweg,  E.E.,  A  Yates ,  D.  Airborne  performance  measurement  methodology 
application  and  validation:  F-4  pop-up  training  evaluation.  AFHRL-TR-79-7.  Williams  AFB, 
AZ:  Flying  Training  Division,  Air  Force  Human  Resources  Laboratory,  April  1979. 


Objective 

The  purpose  of  this  study  was  to  develop  a  system  for  evaluating  performance  on  the  pop-up  weapons 
delivery.  Hie  performance  evaluation  system  was  intended  to  be  applicable  to  a  wide  variety  of  flying 
environments  and  to  be  usable  on  several  types  of  air-to-surface  fighter  aircraft  systems.  Specific 
objectives  of  the  study  were  to  assess  the  validity  and  applicability  of  the  performance  measurement 
system. 


Approach 

The  study  was  begun  by  interviewing  mission-ready  pilots.  The  data  from  these  interviews  were  used  to 
construct  a  rating  form  for  evaluation  of  performance  on  the  pop-up.  On  this  form,  the  pop-up  maneuver 
was  segmented  into  a  sequence  of  critical  stages.  Each  stage  was  associated  with  the  skills  required  to 
support  the  performance  of  that  stage.  Pilot  skill  levels  were  inferred  from  the  performance  data.  Bomb 
score  data  were  also  obtained  for  use  in  assessing  the  validity  of  the  scoring  Bystem. 


Background 

The  present  research  is  a  part  of  the  first  phase  of  the  Air  Force  Skills  Maintenance  and  Reacquisition 
Training  research  program  (Project  SMART).  This  effort  is  aimed  at  defining  flying  skills  critical  to 
mission  readiness  and  developing  procedures  to  assess  these  skills.  Skill  data  will  be  used  to  aid  in 
increasing  the  efficiency  of  Air  Force  continuation  training  programs. 

The  pop-up  weapons  delivery  was  the  first  maneuver  selected  for  study.  Proficiency  at  this  maneuver  is 
critical  to  mission  readiness,  and  the  maneuver  taps  several  areas  of  flying  skill.  In  order  to  perform  the 
pop-up,  a  pilot  needs  to  be  proficient  at  low  altitude  flight  and  navigation,  instrument  cross-check,  and 
basic  fighter  maneuvering,  as  well  as  basic  bombing. 


Specifics 

The  pop-up  performance  rating  form  incorporated  a  nine  (controlled  range  sorties)  or  ten  (tactical 
range  sorties)  stage  segmentation  of  the  maneuver.  Each  stage  was  rated  on  a  three-point  scale 
(Satisfactory  —  Marginal  —  Unsatisfactory).  Explanatory  comments  were  required  when  performance 
on  a  stage  was  rated  than  satisfactory.  Detailed  instructions  on  use  of  the  form  were  provided. 

The  subjects  were  21  F-4  B-course  pilots  assigned  to  the  311th  Tactical  Fighter  Training  Squadron, 
Luke  AFB.  The  subjects  were  undergraduate  pilot  training  graduates  and  had  completed  fighter  lead-in 
training.  AH  had  completed  the  ground  attack  phase  of  training,  but  none  had  previously  performed  the 
pop-up  weapons  delivery  maneuver. 
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Performance  ratings  were  made  by  intructor  pilots  (IPs).  Following  each  sortie,  the  IP  completed  the 
pop-up  weapon  delivery  performance  evaluation  form  from  notes  taken  during  the  flight. 


Results 

Data  were  obtained  on  six  ground  attack  tactical  sorties  (CAT  1  through  CAT  6).  The  first  three  sorties 
(CAT  1  through  CAT  3)  were  performed  on  controlled  ranges.  The  last  three  sorties  (CAT  4  through  CAT 
6)  were  performed  on  tactical  ranges.  Data  were  collected  on  all  CAT  sorties  for  every  pilot. 

In  order  to  validate  the  performance  rating  system,  a  composite  score  was  calculated  for  each  pop-up 
weapon  delivery.  Under  this  system,  the  stages  were  examined  in  sequence.  One  point  was  given  for  each 
satisfactory  stage  rating.  A  marginal  stage  was  given  no  point.  Scoring  of  the  pass  was  terminated  at  the 
first  unsatisfactory  stage,  and  only  stages  preceding  that  stage  were  counted  toward  the  composite  score. 
The  range  of  composite  scores  was  zero  to  nine  (controlled  range)  or  ten  (tactical  range).  Composite  score 
was  used  to  predict  the  bomb  score  category  into  which  the  delivery  would  fall.  Bomb  scores  categories 
were  0  to  49  feet,  50  to  99  feet,  100  to  149  feet,  150  to  199  feet,  200  to  299  feet,  300+  feet,  and  aborted 
pass.  The  composite  rating  score  and  the  bomb  score  were  found  to  be  strongly  related  (chi-square 
contingency  coefficient  =  .66,  p  <  .001).  This  result  demonstrated  the  validity  of  the  rating  system. 

An  analysis  of  the  stages  within  each  pass  was  accomplished  by  computing  the  group  percentage  of 
satisfactory  ratings  given  on  each  stage.  This  analysis  indicated  that  pilots  found  difficulty  with  several  of 
the  stages  of  the  pop-up  weapon  delivery.  The  results  of  the  present  study  suggested  the  following: 

1.  Subjective  ratings  provide  a  valid  and  reliable  measure  of  pilot  performance.  This  was  shown  by 
the  high  contingency  coefficient  obtained  between  the  composite  rating  score  and  bomb  score. 

2.  Ratings  provide  information  on  performance  of  the  individual  stages  of  the  delivery  which  is  not 
available  from  overall  objective  performance  measures,  such  as  bomb  score.  This  assessment  of  stage 
performance  levels  permits  inferences  to  be  made  about  the  overall  effectiveness  of  training  programs. 
Stage  performances  reflect  the  amount  of  training  emphasis  given  to  certain  flying  skill  areas.  By 
identifying  skill  area  proficiency  levels,  training  programs  can  be  modified  to  meet  the  performance 
standards  required  for  mission  readiness. 
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Pierce,  B.J.,  DeMaio,  J.C.,  A  Yates,  D.  Validation  of  an  in-flight  performance  measurement 
methodology:  F-4  ground  attack  training  evaluation.  Paper  presented  at  the  annual  meeting  of 
the  Human  Factors  Society.  Box  ton.  October  1979. 


Background 

Fighter  pilot  training  evaluations  have  traditionally  been  based  on  subjective  instructor  pilot  (IP) 
analysis  of  overall  performances.  In  addition,  during  the  bombing  phases  of  fighter  training  programs, 
criteria  have  been  established  using  bombing  circular  error  for  event  qualification.  Although  these 
measures  are  indicative  of  overall  performance,  they  do  not  provide  detailed  information  necessary  for 
complete  performance  evaluation.  What  is  required  is  a  performance  measurement  system  which  can  be 
used  to  provide  fine  grain  information  about  pilot  performance  on  each  stage  of  a  particular  maneuver.  In 
the  present  study,  detailed  information  about  bombing  performance  was  obtained  by  dividing  the 
conventional  weapon  delivery  maneuver  into  a  sequence  of  stages,  each  of  which  could  be  rated  by  an 
instructor  pilot. 


Approach 

At  the  start  of  the  research,  extensive  interviews  were  conducted  with  mission  ready  pilots  to  identify 
and  define  the  critical  performance  requirements  for  conventional  deliveries.  Discrete  stages  of  the 
delivery  critical  to  overall  performance  were  identified.  Since  means  for  determining  the  values  of 
weapon  delivery  parameters  were  not  avilable  for  use  in  evaluating  airborne  performance,  a  performance 
measurment  methodology  was  developed  which  took  advantage  of  the  IPs'  ability  to  assess  performance 
on  the  critical  stages  of  the  delivery. 

Eight  F-4  B-course  pilots  assigned  to  the  311th  Tactical  Fighter  Training  Squadron,  Luke  AFB,  AZ. 
served  as  subjects.  The  B-course  pilots  are  undergraduate  pilot  training  graduates  who  have  completed 
fighter  lead-in  training  but  have  not  been  qualified  in  any  other  aircraft.  At  the  time  the  study  was 
conducted,  subjects  were  ready  to  begin  ground  attack  (CA)  training.  Four  weapon  delivery  maneuvers 
were  taught;  10  degree  low  angle  bomb.  20  degree  low  angle/low  drag  bomb.  30  degree  dive  bomb,  and  45 
degree  dive  bomb.  The  number  of  deliveries  performed  for  each  maneuver  varied  to  match  the  pilots" 
need  for  practice.  In  no  case  were  more  than  four  maneuvers  performed  on  one  sortie.  Generally,  each 
pilot  performed  12  deliveries  on  each  sortie. 

Pilot  performance  ratings  were  collected  using  the  F-4  GA  Phase  Evaluation  form.  The  form  segmented 
the  delivery  into  six  critical  stages,  (1)  baseleg  positioning,  (2)  roll-in,  (3)  roll-out,  (4)  pipper  placement. 
(5)  bomb  run,  and  (6)  recovery.  The  baseleg  and  bomb  run  stages  were  further  subdivided  into 
component  tasks.  Performance  on  each  stage  and  component  task  was  rated  using  a  three-point  scale 
(Satisfactory-Marginal- Unsatisfactory ).  Explanatory  comments  were  required  for  all  marginal  and 
unsatisfactory  ratings.  Bomb  scores  and  estimated  bomb  release  parameters  were  recorded  for  each 
delivery.  Pilot  performance  was  assessed  and  recorded  by  IPs  at  the  debriefing  of  each  mission. 
Performance  ratings  were  collected  on  all  deliveries  performed  by  the  subject  pilots  throughout  the  GA 
phase. 


Results  and  Discussion 

An  overall  composite  rating  of  each  pass  was  generated  from  the  ratings  of  the  five  individual  stages 
occurring  prior  to  weapons  release.  The  relationship  between  the  composite  rating  score  and  bomb  score 
was  assessed  by  a  X  contingency  analysis.  The  composite  rating  score  was  found  to  vary  directly  with 
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bomb  score  (\  =  198.3,  P  <  .001;  C  =  .5).  These  results  demonstrate  the  validity  of  the  rating 

methodology  by  relating  the  ratings  to  the  best  available  objective  measure  of  the  quality  of  a  delivery 
(i.e..  bomb  score).  The  pilots'  progress  over  the  course  of  training  was  expected  to  improve  over  the  course 
of  the  CA  phase,  and  the  data  supported  this  expectation. 

Chi-square  contingency  analyses  were  performed  on  the  ratings  of  the  individual  stages  to  determine 
the  strength  of  their  relationship  to  bombing  accuracy.  The  stages  which  were  found  to  bear  the  strongest 
relationship  to  bomb  scores  were  stage  4,  pipper  placement,  and  stage  5,  bomb  run.  These  results 
demonstrate  that  while  the  correct  initial  setup  of  the  delivery  is  necessary,  it  is  the  final  stages  that  are 
most  critical  to  bombing  accuracy. 

The  present  research  has  supported  the  conclusion  of  Pierce,  DeMaio,  Eddowes,  and  Yates,  1979,  that 
instructor  pilot  ratings  can  provide  valid  fine  grain  information  about  pilot  performance. 
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APPENDIX  D:  SUMMARY  REPORT:  LOW  ALTITUDE  TACTICAL 
FORMATION  SELF-ASSESSMENT  EVALUATION  STUDY 


DeMaio,  J.C.,  A  Eddowes,  E.E.  Airborne  performance  measurement  assessment:  Low  altitude  tactical 
formation  in  two  operating  environments.  AFHRL-TR-79-44.  Williams  AFB,  AZ:  Flying 
Training  Division,  Air  Force  Human  Resources  Laboratory,  February  1980. 

Objective 

A  study  was  conducted  to  determine  what  areas  of  skills  are  required  for  successful  low  altitude  tactical 
formation  (LATF)  flying.  It  also  investigated  how  well  routine  flying  prepared  pilots  for  flying  in  a 
simulated  high  threat  environment.  • 

Approach 

The  study  was  begun  by  interviewing  mission-ready  pilots.  Data  from  these  interviews  were  used  to 
construct  a  rating  form  for  evaluation  of  low  altitude  tactical  formation  flying  performance.  The 
importance  of  pilot  skills  was  inferred  from  a  multiple  regression  analysis  in  which  the  rating  of  overall 
performance  was  predicted  from  performance  on  the  various  skills. 

Background 

The  present  research  is  a  part  of  the  first  phase  of  the  Air  Force  Skills  Maintenance  and  Reacquisition 
Training  research  program  (Project  SMART).  This  effort  is  aimed  at  defining  flying  skills  critical  to 
mission  readiness  and  developing  procedures  to  assess  these  skills.  Skill  data  will  be  used  to  aid  in 
increasing  the  efficiency  of  the  Air  Force  continuation  training  programs.  The  LATF  was  the  second 
tactical  flying  task  selected  for  study.  Proficiency  at  this  task  is  critical  to  mission  readiness,  and  the  task 
taps  several  areas  of  flying  skill. 

Specifics 

The  low  altitude  tactical  formation  performance  evaluation  form  included  eitht-point  rating  scales  for 
evaluating  performance  on  the  following  flying  skills:  visual  lookout,  maintaining  position  (own 
performance),  maintaining  position  (element  member’s  performance).  Time  on  Target,  distance  off 
target,  and  overall  performance.  In  addition,  a  rating  of  Familiarity  with  element  member  (eight-point 
scale)  was  included.  Low  altitude  flying  performance  data  were  comfort  level  and  three  measures  of 
minimum  altitude  capability  (flying  straight  and  level,  over  rough  terrain,  and  in  turns).  These  were 
given  in  feet  above  ground  level. 

Data  were  collected  in  two  operating  environments.  Twenty -eight  A-7  pilots  of  the  354  Tactical  Fighter 
Squadron  at  Davis-Monthan  AFB  provided  self-rating  data  on  their  formation  flying  performance  during 
routine  operations.  Twenty  of  these  same  pilots  provided  data  on  their  performance  in  a  simulated  high- 
threat  environment  (Red  Flag). 

Data  were  analysed  via  a  step-wise  linear  multiple  regression  to  determine  which  pilot  performance 
variables  were  important  to  pilots’  ratings  of  their  overall  formation  flying  performance.  Variables 
examined  were  related  to  (a)  the  mutual  support  role  of  members  of  the  formation  in  defending  the 


22 


formation  from  attack,  (b)  low  altitude  flying  performance,  and  (c)  navigation  performance.  Four 
variables  were  found  to  provide  a  reliable  index  of  overall  individual  LATF  performance  (R  =  .92)  in  the 
routine  operating  environment.  These  were  (a)  visual  lookout,  clearing  the  area  for  threats,  (b) 
maintaining  position  in  the  formation  (own  performance),  (c)  maintaining  position  in  the  formation 
(element  member’s  performance),  and  (d)  familiarity  with  element  member.  The  first  two  variables  were 
felt  to  be  directly  related  to  the  integrity  and  survivability  of  the  formation.  The  last  two  variables,  which 
did  not  reflect  the  individual's  own  performance,  were  felt  to  be  important  in  determining  pilot  workload. 

The  regression  analysis  of  performance  in  the  high-threat  environment  revealed  important 
differences  from  the  routine  operating  environment.  In  particular,  the  importance  of  visual  lookout 
increased  by  60%.  Familiarity  with  element  member  ceased  to  be  important  in  the  high-threat 
environment.  This  was  interpreted  to  reflect  a  lack  of  consistency  in  pilot  behavior  in  the  routine  and 
high-threat  environments. 


Conclusions 

1.  Pilot  performance  self-rating  methodology  was  successfully  applied  to  the  low  altitude  tactical 
formation  task.  Pilot  performance  ratings  were  found  to  have  high  consistency. 

2.  Skill  areas  found  to  be  important  to  the  quality  of  an  individual’s  formation  performance  were 
related  to  the  mutual  support  role  of  the  formation.  Specific  skill  areas  were  (a)  visual  lookout,  clearing 
the  area  for  threats  and  (b)  maintaining  position  in  the  formation. 

3.  An  individual’s  performance  rating  was  found  to  be  affected  by  the  other  member  of  his 
element.  An  individual's  performance  rating  increased  when  his  rating  of  his  element  member’s 
performance  improved.  Also,  individuals  rated  their  performance  higher  when  they  flew  with  a  familiar 
element  member. 

4.  When  placed  in  a  simulated  combat  environment,  pilots  were  found  to  experience  a  period  of 
reorientation  and  atypical  behavior  similar  to  that  reportedly  experienced  in  the  actual  combat  situation. 
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APPENDIX  E:  SUMMARY  REPORT:  TAC-WIDE  TEST 


Lyon,  D.R.,  Eubanks,  J.L.,  Killion,  T.H.,  Nullmeyer,  R.T.,  &  Eddowes,  E.E.  Pop-up  weapon-delivery 
maneuver:  Use  of  pilot  self-assessment  data  in  analysis  of  critical  components.  AFHRL-TR-80- 
33.  Williams  AFB,  AZ:  Operations  Training  Division.  Air  Force  Human  Resources  Laboratory. 
October  1980. 


Objective 

At  the  request  of  Tactical  Air  Command  Headquarters,  a  test  of  an  aircrew  self-report  methodology  for 
isolating  the  critical  components  of  complex  skills  was  conducted  at  mission-ready  F-4  squadron  across  the 
United  States.  This  methodology  was  being  explored  for  possible  use  by  the  Air  Force  Skill  Maintenance 
and  Reacquisition  Training  Program  (Project  SMART),  which  had  the  goal  of  defining  and  measuring  the 
most  critical  skills  supporting  aircrew  mission  readiness.  The  pop-up  weapon-delivery  maneuver  was 
chosen  as  one  of  the  test  cases  for  determining  the  usefulness  of  the  methodology. 


Approach 

Aircrews  provided  structured  assessments  of  their  performance  on  scored  pop-up  weapon  deliveries  in 
the  following  manner.  The  maneuver  was  subdivided  into  eight  pre-delivery  components  and  the 
parameters  which  define  excellent,  satisfactory,  marginal,  and  unsatisfactory  performance  on  each  were 
given  to  aircrews  after  each  delivery.  Aircrews  rated  their  performance  on  each  component  of  the  delivery 
and  listed  errors  which  caused  less  than  satisfactory  performance.  Researchers  stressed  that  this 
information  would  not  be  used  for  evaluating  in  lividuals  or  squadrons. 

Several  analyses  were  performed  on  assessment  and  listed  errors,  in  order  to  determine  the  rel  itiv  - 
contribution  of  each  component  to  delivery  accuracy. 


Specifics 

Pilots  and  Weapons  Systems  Officers  (WSO)  from  nine  operational  F-4  squadrons  participated  in  the 
study.  Squadrons  participating  were  those  of  the  474th  Tactical  Fighter  Wing  (TFW)  at  Nellis  AFB.  the 
347th  TFW  of  Moody  AFB.  and  the  4th  TFW  at  Seymour  Johnson  AFB.  Performance  assessments  were 
gathered  during  a  1 -month  period  at  each  squadron,  and  in  all.  data  were  obtained  for  545  deliveries.  The 
following  analyses  were  performed  on  these  data: 

1.  Component  assessments  and  weapon  delivery  accuracy  category  were  cross-classified.  Separate 
analyses  were  performed  for  different  delivery  events  and  for  pilot  and  WSO  data. 

2.  The  strength  of  the  relationship  between  performance  assessments  on  each  delivery  component  and 
delivery  accuracy  was  determined.  The  obtained  pattern  of  relationships  was  analyzed  further  to  evaluate 
several  hypotheses  about  its  cause. 

3.  Reported  delivery  errors  were  also  partitioned  according  to  the  delivery  component  on  which  they 
occurred.  Conditional  probabilities  of  a  failed  delivery,  given  that  an  error  was  reported,  were  computed 
for  different  parts  of  the  delivery. 
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Results 


The  major  result  of  the  analysis  of  both  the  rating  data  and  the  reported  error  data  was  that  the  final  two 
componen..  of  the  delivery  (track  point  and  bomb  run)  showed  by  far  the  strongest  relationships  to 
delivery  accuracy.  This  was  true  for  all  delivery  events  examined  and  for  both  pilot  and  WSO  data.  No 
consistent  relationships  between  earlier  delivery  components  and  accuracy  were  found. 

This  result  was  apparently  not  a  consequence  of  a  cumulation  of  errors  in  earlier  components  leading  to 
predictive  track  point  and  bomb  run  performance.  The  evidence  for  this  is  that  the  track  point  and  bomb 
run  performance  remain  predictive  of  accuracy  for  a  subset  of  the  data  in  which  all  earlier  components 
received  satisfactory  or  excellent  ratings.  Further  analyses  showed  that: 

1.  Track  point  and  bomb  run  ratings  are  not  independent  predictors  of  accuracy.  For  example,  the 
partial  correlation  between  the  track  point  ratings  and  accuracy,  with  the  bomb  run  ratings  held  constant, 
is  non-significant. 

2.  Two  potential  indices  of  component  difficulty,  proportion  of  marginal  and  unsatisfactory  ratings 
and  number  of  reported  errors,  suggest  the  tract  point  and  bomb  run  are  the  most  difficult  parts  of  the 
maneuver  for  these  experienced  crews. 

3.  The  most  frequent  error  reported  for  track  point  and  bomb  run  components  was  failure  to  correct 
adequately  for  wind. 

It  should  be  noted  the  results  apply  only  to  experienced  aircrews  on  training  ranges,  where  navigation 
problems  are  minimal. 


Conclusions 

The  results  suggest  that  experienced  aircrews  can  execute  the  initial  parts  of  the  pop-up  maneuver  well 
enough  that  errors  in  this  portion  of  the  maneuver  do  not  greatly  affect  bomb  scores.  For  these  pilots,  most 
of  the  value  of  pop-up  training  drills  may  consist  of  the  opportunity  to  practice  the  final  seconds  of  the 
maneuver. 

The  overall  evaluation  of  this  methodology  is  that,  while  it  is  clearly  no  substitute  for  direct  measures  of 
components  of  performance  when  available,  it  can  nevertheless  yield  useful  results.  Some  difficulties  with 
use  of  self-assessment  were  surmountable  when  ratings  of  components  were  obtained. 
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